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ABSTRACT 

Every teacher understands the importance of getting timely student feedback for effective and 
efficient teaching and learning. However, students are not always keen to answer questions in the 
classroom in front of their peers. There is a need for an efficient method to engage all the 
students in a classroom and quickly evaluate the progress of their learning. 

Student response systems (SRS) are effective increasing student engagement, even in a large 
lecture hall. In a traditional SRS, the students use a small portal device, called a clicker, to 
choose their answers. Then proprietary hardware and software will collect and display the results 
to the class. With the widespread use of mobile phones, it is possible to replace the clickers with 
mobile phones, and the proprietary software with commercially available polling website. 

This study used a polling website and combined it with the mobile phones of students to form an 
SRS. The SRS was for 6 weeks and 1,155 answers were received. A survey afterwards showed 
that the students had very positive attitude towards the SRS. The majority of the students 
indicated that SRS made the lessons more interesting and helped them to maintain their attention. 
They also indicate that they were willing to use SRS in future. 

Keywords: Student Response System; Mobile Learning; Polling; Clickers; Smart Phone; 
Learning Management System; Class Response System; Audience Response System 


INTRODUCTION 

To achieve effective and efficient learning, student engagement is essential, but not easy to 
achieve (Micheletto, 2011; Wang, Shen, Novak, & Pan, 2009). In a classroom where the majority 
of the students are Chinese, the lack of student engagement is often an obstacle to achieving the 
learning outcomes (Wang et al., 2009). As digital technology continues to improve and become 
more economically viable to schools, many researches have been done to exploit technology to 
increase the student engagement (Hwang, Wu, Tseng, & Huang, 2011; Jungsun & Kizildag, 2011; 
Liu & Chen, 2015). Recently, many researches have focused on the use of SRS (Student 
Response Systems) in which the teacher can gather and summarise answers from students 
inside the classroom immediately (Carnaghan, Edmonds, Lechner, & Olds, 2011; Seamus 
McLoone, Villing, & O'Keeffe, 2015; Monk, Campbell, & Smala, 2013; Valle & Douglass, 2014). In 
a traditional SRS, the students use a small portal device, called a clicker, to choose their answers 
to questions. Then some proprietary software with summarise the responses from the students 
and display them on screen (Williams & Boyle, 2008). 

Many studies revealed that SRS are effective increasing student engaging and active learning 
(Cain, Black, & Rohr, 2009; Lindquist et al., 2007; Park, Nam, & Cha, 2012; §ad & Gokta§, 2014). 
They are especially useful in creating a more engaging environment in a large lecture hall. The 
main benefit of using an SRS is that students are not afraid to answer questions because 
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students can only see the statistics of the different answers, but not the individuals who gave the 
answers. However, using clickers means the school or the students have to purchase the 
hardware and software (Monk et al., 2013). Also, the clickers are usually small numeric keypads 
that have limited text entry capabilities. With the widespread use of smart mobile phones, it is 
possible to replace the clickers with mobile phones, and the proprietary software with 
commercially available polling website. The resulting system can be called a “mobile phone 
based SRS”. 

If the mobile phone based SRS is technically feasible in a school and if students have positive 
perceptions of using such a system, then teachers can use it in class to increase student 
engagement and to identify individual students who are underperforming. Therefore, the purpose 
of this study is to perform a pilot study at a particular school, and then survey student perceptions 
to provide a reference to teachers at the same school for their decisions to use the system or not. 
The rest of this article is structured as follows. It starts with by reviewing the advantages mobile 
phones over the traditional methods of getting student feedback in the classroom. Then it reports 
the study that using an SRS that was created by combining a polling website with mobile phones 
of the students. Finally, it shows the results of a survey about the student's perception of the use 
of SRS. Lastly, some suggestions for future research in this area of mobile learning are 
suggested. 


MOBILE PHONES BASED SRS VERSUS TRADITIONAL QUESTIONING METHODS 

In the tradition classroom where no technology is used, the teacher cannot get real-time feedback 
from all the students in the classroom. Typically, the teacher will ask some questions and request 
students to answer them verbally or raise their hands to choose their answers. Alternatively, the 
teacher may ask students to write down their answers on pieces of paper. These traditional 
methods have been used over a long period of time and can be used across many disciplines. 
Since no technology is involved, there is no computer knowledge or equipment requirement. 
However, this traditional method has some shortcomings that greatly reduce its effectiveness, 
especially in large lecture theatres. 

When the teacher wants to get qualitative answers from students, they would invite students to 
answer open-ended questions verbally. Many students are too shy to give verbal answers in front 
of their classmates (Wang et al., 2009). Therefore, some students who are extroverts will be 
responding to questions frequently, and the teacher cannot get answers from the shy students. It 
is possible that the shy students may have some creative and original answers that the whole 
class can benefit from. Moreover, it is not uncommon that when students try to answer questions 
verbally, their voices are so low that the teacher and the rest of the class cannot hear them 
clearly. This problem becomes more serious in Hong Kong where classes are often conducted in 
English, but the student's mother tongue is Chinese. If students can use mobile phones to 
provide answers without revealing their identities to the whole class, even shy students are not 
intimated to answer questions and show their answers in front of the class. 

When the teacher wants to get quantitative answers from students, they would invite students to 
raise their hands to indicate their choice of answers to close-ended questions. If the teachers ask 
the students to raise their hands, many of them are afraid to be the first ones. Conversely, when 
most of the students raised their hands, the remaining few students are often forced to raise their 
hands too (Withey, 2010). Raising hands creates the problem for the teacher because it is hard to 
count and even harder to keep track of who raise their hands to what question. Therefore, the 
raise-of-hand method can only provide a general impression of student's understanding of the 
contents, but cannot be very effective in identifying and tracking the progress of students who 
have misconceptions. If the teacher collects answers in written form in class, then the teacher has 
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the extra effort of handling paper and try to interpret styles of writing that are sometimes difficult 
to read (Bae & Kim, 2014; Cheung, 2008). 

The limitations of the traditional methods stated above mean the teachers who teach the same 
subject cannot easily and accurately compare the response and progress of the students. With a 
mobile phone based SRS, the teacher can collect and show answers instantaneous on screen. 
Students will not feel anxious to answer questions because students can only see the statistics of 
the answers, but not the names of students who gave the answers. However, the teacher can 
track the answers to individual students using reports provided by the website. The answers are 
stored on the server and can be analysed by the teacher later. There is no extra effort to count 
hands and handle paper. The teacher can focus on the teaching. The statistics help the teacher 
to pin point the students who have misconceptions and, therefore, need more help. The statistics 
from different classes on the same subject can be collated and compared easily. However, the 
use of a mobile phone based SRS does require certain technical skills on the teacher, the cost of 
subscription to a polling website and having a good Wi-Fi coverage in the classrooms involved. 
Furthermore, there is a possibility that students may become less responsive when traditional 
questioning methods are used. 


THE STUDY 

The study was carried out at the School of Professional Education & Executive Development of 
the Hong Kong Polytechnic University. The subjects of the study were full-time students who 
were at their first-year of a two-year top-up degree programme in information technology. After 
completing the top-up degree programme, the students would get their bachelor degree in 
Information Systems and Web Technologies. 

The author used PollEverywhere, a commercially available polling website to teach a class of 
undergraduates who were taking the Management Information Systems course in Hong Kong. 
The author also compared PollEverywhere with similar websites such as Socrative. The website 
was chosen based on cost considerations and the fact that the author learned to use it in a 
seminar about teaching and learning. PollEverywhere can accept student response in a number 
of channels including SMS text, Twitter and Web. This study only accepted student responses 
through the web because of the following reasons. Firstly, SMS text may have cost implications 
for the students, who are studying full-time. Secondly, Twitter is not popular among the students 
in this school. 

There are no costs to the students who are subjects of this study. The campus at which the 
author is teaching provides free WIFI access to all students via both the Eduroam and CPCE- 
Student. This means students do not need to subscribe mobile data plans for answering polls in 
this course. The software offered a free trial version which can only accept 25 responses per poll 
for business users. For educators, the software offered a free version which can accept 40 
responses per poll. In both trial versions, grading of responses is not allowed. 

The cost of the software is minimal and flexible. The author subscribed the USD19 per month 
plan which can accept up to 50 responses and allowed grading of responses. It is flexible 
because the plan can be terminated when polling is no longer required. In terms of question 
design, the author’s subscribed version and higher version is the same. The more expensive 
versions have additional features such as moderation of answers, more responses per question 
and team competition. In some lessons, the author found it necessary to create questions 
immediately during the lesson. It took the author less than one minute to create a question and 
then poll the students for answers. 
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Student accounts are created without too much effort. The software can import student accounts 
directly from Management Systems (LMS) such as Blackboard and Canvas. Since the LMS is not 
supported by the software, the author created forty-four students accounts exporting student 
names from the School’s LMS into a comma-separate value (CSV) file. Then they are formatted 
according to the requirements of the polling website. Finally, they are imported into the polling 
website to create usernames and passwords. 

The SRS was used in 12 lessons over a period of 6 weeks. Thirty-five questions were asked and 
1,155 answers were received. There are three types of questions, namely, true/false, multiple 
choice and open-ended. On average, three questions were asked through polling per lesson. The 
maximum number of questions used in one class was six. The student responses were exported 
from the PollEverywhere website and then imported into Microsoft Excel for analysis. The 
averages and standard deviations were calculated. 

At the end of the study, a survey was conducted using an online questionnaire. The survey was 
performed in a computer lab in which the students filled out the questionnaire anonymously on a 
website which is different from the polling website. There were forty-four students in the class and 
thirty-nine students completed the questionnaire. The survey response rate was 89%. The results 
of the survey are presented and discussed in the following section. 


FINDINGS 

Both the hardware and the polling website worked smoothly during the study. The question 
creation process requires about one full day to learn. It was found that the majority (85%) 
students used mobile phones as the device for answer questions in the polls. Only a few students 
used the tablet (5%) and notebook computer (10%) to answer questions in the polls (Table 1). 
The phone reception was excellent in the classroom and no students experience problems in 
connectivity. Three students created their own guest accounts when they didn't have to. They 
answered questions using the wrong account until 7 out of the 33 polls are done. 


Table 1 Devices used by students for answering questions in the polls 


Device Type 

Number of Students 

Percentage (n=39) 

Mobile phone 

33 

85% 

Tablet 

2 

5% 

Notebook Computer 

4 

10% 


The response rate was satisfactory (Table 2). Students are more responsive to true/false 
questions, which have the highest response rate of 68%. Students are also willing to answer 
multiple choice questions, whose response rate is 56%. However, less than half of the students 
responded to open-ended questions. Note that the response rate is the percentage of student 
responses divided by students on the register, not students in the classroom. Since some 
students in the register may not attend all lessons, the response rate may be underestimated. 
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Table 2 Response rates to different types of poll questions 


Question Type 

Response Rate* 

Number of Polls 

Multiple Choice 

56% 

16 

Open-Ended 

47% 

5 

True/False 

68% 

2 


The open-ended questions were useful because it is easy to spot the problems the students 
faced. There are two problems that occurred frequently. Firstly, many students did not follow the 
instructions and they used more words than the question allowed. Secondly, there are 
grammatical and/or spelling mistakes in their answers. For example, a student wrote down 
“willing to be promotion”. No offensive or obscene answers were received for open-ended 
questions. This is probably because it was stated clearly at the beginning that the teacher could 
track the users to the students who made them. 

The students were excited when the polling website showed statistics about the answers 
immediately and presented them in various formats. For the software being used by the author, 
four formats were possible. For true/false and multiple choice questions, the statistics of the 
answers can only be shown as bar charts (Figure 1). For open-ended questions, the answer can 
be presented on a word cloud or cluster (Figures 2 to 3). The students' answers to the questions 
about their perceptions of SRS are summarised in Table 3. 


Polls Reports Participants • 



L02h - Choose the right names to the bubble diagram in th / 
order of 1,2,3, 4 and 5. .,= 
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Figure 1: Statistics of student responses to a multiple-choice question shown as bar chart 
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L03i Name three of the forces in Porter's Model. 
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Figure 2: Student responses to an open-ended question shown as a word cloud 


L03i Name three of the forces in Porter's Model. 


a Poll is full and no longer accepting responses 

“supplier bargaining power" “suppliers bargaining power” “barrier to new entrants” “Customers' bargaining” 


□ 


“Threat of substitute products and service" 


“Customers’ bargaining power" 


“Supplier, customers, traditional competitors” 


“Competition among existing competitors” “customer's bargaining power" “customer bargaining power” “Barrier to new entrants” “Barrier to new entrants” 

“customers bargaining power” “Competition among existing competitors” “Barriers for new entrants” "Customer's bargaining power" “Barrier to new entrants” 


“Barrier's Resonse” “supplier bargaining power" “Suppliers' bargaining power” “customer's bargaining power" “customers bargaining power" 


“Customer's bargaining power, Supplier's bargaining power, Barrier to new entrats" “Supplier's bargaining power” 


Figure 3: Student responses to the same open-ended question as in Figure 2, but shown as 
clusters. 
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Table 3: Student Perception on the SRS 

(“1” to “5” represents Strongly Disagree, Disagree, Neutral, Disagree, and Strongly Disagree respectively) 



Statement 

Average 

Score 

Standard 

Deviation 

%age of 
“Strongly 
Agree” or 
“Agree” 

Q1 

My mobile device(s) work(s) well with the polling 
website. 

4.28 

0.7493 

95% 

Q2 

1 can access the polling website easily. 

4.13 

0.8221 

90% 

Q3 

Learning with SRS is relevant to the subject of 
Information Systems. 

4.13 

0.8221 

85% 

Q4 

Answering questions using SRS helps me to 
maintain my attention. 

4.10 

0.9280 

74% 

Q5 

Answering questions using SRS makes the 
lessons more interesting. 

4.10 

0.8999 

77% 

Q6 

1 have the necessary skills for answering 
questions using SRS. 

4.18 

0.7467 

92% 

Q7 

It is easy to answer questions using SRS. 

4.10 

0.8410 

82% 

Q8 

1 am willing to answer questions using SRS in 
future. 

3.95 

0.9858 

85% 

Q9 

If another lecturer uses SRS, 1 will answer all 
questions. 

3.95 

0.9322 

72% 

Q10 

1 like answering questions using SRS. 

3.87 

0.9917 

67% 


DISCUSSION 

The following paragraphs discuss the student answers to the questions Q1 to Q10 listed in the 
table above. These questions can be divided into five groups. Each group will be discussed in a 
separate paragraph below. 

Firstly, the students were happy with the hardware and the website that form the SRS. The 
mobile device ownership was 100% and all students were able to participate in the SRS using 
their mobile devices. This is consistent with similar research in other universities. For Q1 and Q2, 
almost all students said their devices worked well with the SRS, and that they could access the 
polling website easily. The scores were 4.28 and 4.13 respectively. 

Secondly, the student had positive perception in the usefulness of SRS in helping them to learn 
the subject. For Q3, Q4 and Q5, the majority of the students also agreed that SRS is relevant to 
the subject, and that the SRS helped them to maintain their attention and made the lessons more 
interesting. The scores were 4.13, 4.10 and 4.10 respectively. It was probably because the 
answers from the students were shown in attractive ways such as bar charts, word clouds and 
clusters. It was also because some students gave unusual answers to some questions. 

Thirdly, the students indicated that they didn't need much time and effort to learn to use the SRS. 
For Q6 and Q7, the majority of the students said they had the necessary skills to use SRS and it 
was easy to answer questions using SRS. The scores were 4.18 and 4.10 respectively. It was 
because the students only need to enter the URL of the polling website into their mobile device 
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and log in once in the first lesson. In subsequent lessons, the students didn't have to log in again. 
Also, because the teacher could control which exact question the students saw so that they didn’t 
have to navigate around to find the right question to answer. 

Fourthly, although the students were willing to use SRS in future, the average scores were lower 
than the three groups of questions above. For Q8 and Q9, the majority of the students mentioned 
that they were willing to use SRS in future and if another lecturer uses SRS, they would answer 
all questions. The scores were 3.95 and 3.95 for both questions, whereas the scores for 
questions Q1 to Q7 were all above 4. There are two possible reasons. The first possible reason is 
that the students in this study had no previous experience with SRS. Therefore, they would need 
more time to be comfortable and natural at it. The second possible reason is that the author was 
also new to such a system, and the use of the system might not have been well-integrated into 
the teaching of the subject. 

Finally, the students indicated that they like answering questions using SRS. For Q10, the 
average score is 3.87, the lowest among the five groups of questions. The raw data showed that 
26% of the students chose “Neutral” as the answer, while 5% chose “Disagree” and 3% chose 
“Strongly Disagree” to this statement. Therefore, more research is needed to find out why only 
sixty-seven per cent of the students liked answering questions using SRS. One possibility was 
that the polling website that was used in the study does not allow students to change their 
answers after submission. In more than one incident, students indicated that they submitted an 
answer that they didn't intend to. Another possibility was that the students didn't like typing in long 
sentences as answers. 

Because this is an exploratory study, it is limited by the experience of the author with SRS. Also, 
there are other SRS that are available to students studying at certain universities at Flong Kong, 
but not available to the student in the author’s university. One example is uReply, which was 
developed by the Chinese University of Flong Kong. 


CONCLUSIONS AND SUGGESTIONS FOR FUTURE RESEARCH 

In summary, this exploratory study shows that it is feasible to combine student mobile phones 
and polling website to create an effective SRS. The technical side of the SRS went well during 
the study. Students also agreed that it made them more attentive and the lessons more 
interesting. Students had generally very positive perception about the SRS, although more 
research is needed to understand why their intention to use SRS in future is not as positive as 
other perceptions of SRS. Seamus McLoone and Brennan (2013) did a study in mobile phone 
based SRS, but they used purpose-built software instead of commercially available polling 
websites. Flowever, the results from this study are largely in line with theirs. One special finding is 
that the average scores on student's intention to use SRS in future and their general acceptance 
on SRS are lower than the other constructs. Therefore, there is a need to research on the factors 
that affect the student's intention to use SRS in future. For example, if the same cohort of 
students were to use SRS in the next semester, their perceptions may change. Also, if the 
teacher becomes more experienced with the use of SRS, the students may perceive its use more 
positively. 

With the feasibility confirmed, it is suggested that the research on SRS be conducted for large 
lecture classes. It is also suggested that research be conducted on students whose major is not 
information technology related. 

The SRS is an effective and efficient way to get real-time feedback from all students. Flowever, 
the students would not benefit from it unless teachers are willing to adopt SRS as part of their 
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teaching. Therefore, it is suggested that the Technology Acceptance Model (Venkatesh & Davis, 
2000) be adapted to find out the factors that affect the teacher’s intention to use SRS. 
Furthermore, it has been shown that Internet technology such as online surveys can be an 
effective tool to perform peer evaluation (Wong & Ng, 2005). Therefore, one possible use of the 
SRS is to allow students to perform peer evaluations on each other’s work. 
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